System and method for next hop BGP routing in a network

ABSTRACT

Aspects of the present disclosure involve systems for providing multiple egress routes from a telecommunications network for a client of the network. In general, the system provides for a client of the network to receive intended packets of information through multiple connections to the network such that load balancing and failover services for traffic to the customer are provided. The process and system allow for telecommunications network to utilize a common next-hop value of announced border gateway protocol (BGP) routes to advertise multiple routes to reach a destination customer network or address. By utilizing a common next-hop value in the announced BGP information, the devices of the network may load balance communication packets to the destination customer or address among the multiple egress locations from the network, as well as providing fast failover to alternate routes when a failure at the network or customer occurs.

TECHNICAL FIELD

Embodiments of the present invention generally relate to systems andmethods for implementing a telecommunications network, and morespecifically for routing packets within a network based on a next-hopvalue in border gateway protocol (BGP) information that is unique to agateway, a customer, or a local-customer of the network.

BACKGROUND

Telecommunication networks provide for the transmission of informationacross some distance through terrestrial, wireless or satellitecommunication networks. Such communications may involve voice, data ormultimedia information, among others. In addition, telecommunicationnetworks often offer features and/or services to the customers of thenetwork that provide flexible and varied ways in which thecommunications are transmitted over the network. For example, sometelecommunication networks provide Internet access to its customers,long distance communication capabilities, high definition audio and/orvideo communication capabilities, and the like. In other examples, thetelecommunication network may be utilized by other networks to providecertain services to customers of the other networks. In this manner, atelecommunication network may provide any number of communicationservices to the various customers purchasing those services from thenetwork.

To transmit these messages and/or services, many telecommunicationsnetworks include a process of announcing routing information to deviceswithin and outside the network that is then re-broadcast or announced toother devices within or outside the network. This routing informationgenerally provides a route or other transmission information to thenetwork devices on how to reach a destination device or address, whetherinternal or external to the network such that a transmission path froman originating device to a destination device may be created. Throughthis route announcement process, devices within the telecommunicationsnetwork receive an understanding of the connectivity of the networkthrough which communication packets may be transmitted. In someinstances, however, the route announcement process may cause undesirableissues and inefficiencies for the telecommunications network, especiallyfor communication packets intended for border networks or othercustomers that connect to the telecommunications network in multiplelocations and/or connections.

SUMMARY

One implementation of the present disclosure may take the form of amethod for routing a communication through a telecommunications network.The method may include the operations of establishing, utilizing a firstnetwork edge device, a border gateway protocol (BGP) session with afirst edge device of a border network in communication with thetelecommunications network, the BGP session providing first routinginformation for transmitting communication packets to a destinationaddress through the first network edge device of the border network andreceiving, utilizing a second network edge device, second routinginformation from a second edge device of the border network incommunication with the telecommunications network, the second routinginformation for transmitting communication packets to the destinationaddress through the second network edge device of the border network.The method may further include the operation of transmitting internalBGP information from the first network edge device and the secondnetwork edge device to at least one device of the telecommunicationsnetwork, the internal BGP information comprising the destination addressand a group-specific next-hop identifier, the group-specific next-hopidentifier utilized by the telecommunications network to transmitcommunication packets to the border network.

Another implementation of the present disclosure may take the form of atelecommunications network. The network may include a first network edgedevice in communication with a first border network edge device of aborder network, the first network edge device receiving first routinginformation for transmitting communication packets to a destinationaddress through the first border network edge device of the bordernetwork, a second network edge device in communication with a secondborder network edge device of the border network, the second networkedge device receiving second routing information for transmittingcommunication packets to the destination address through the secondborder network edge device of the border network, and at least onerouting device receiving internal routing information from the firstnetwork edge device and the second network edge device, the internalrouting information comprising the destination address and agroup-specific next-hop identifier, the group-specific next-hopidentifier utilized by the telecommunications network to transmitcommunication packets to the border network.

Yet another implementation of the present disclosure may take the formof a networking device including at least one communication port forreceiving routing information associated with a telecommunicationsnetwork, a processing device, and a computer-readable medium connectedto the processing device configured to store information andinstructions. When the instructions are executed by the processingdevice, the networking device is configured to receive internal BorderGateway Protocol (BGP) information from a first network edge device incommunication with a first border network edge device of a bordernetwork to the telecommunications network and a second border networkedge device in communication with a second border network edge device ofthe border network, the internal BGP information comprising adestination address and a group-specific next-hop identifier utilized bythe telecommunications network to transmit communication packets to theborder network. The networking device is further configured to select apreferred route through the telecommunications network to thedestination address based on the group-specific next-hop identifierreceived from the first network edge device and the second network edgedevice and transmit at least one communication packet to the destinationaddress based on the selected preferred route through thetelecommunications network.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematic diagram illustrating an exemplary Internet Protocol(IP) operating environment in accordance with one embodiment.

FIG. 2 is a schematic diagram illustrating a network environment toprovide multiple routes to a client network through the networkenvironment.

FIG. 3 is a flowchart illustrating a method for a provider edge of atelecommunications network to broadcast border gateway protocol (BGP)with a group-specific next-hop value.

FIG. 4 is a flowchart illustrating a method for component of atelecommunications network to transmit a packet to a customer networkbased on a group-specific next-hop value in broadcast BGP information.

FIG. 5 is a diagram illustrating an example of a computing system whichmay be used in implementing embodiments of the present disclosure.

DETAILED DESCRIPTION

Aspects of the present disclosure involve systems, methods, computerprogram products, and the like, for providing multiple egress routesfrom a telecommunications network for a client of the network. Ingeneral, the process allows for a client of the network to receiveintended packets of information through multiple connections to thenetwork, thereby providing a load balancing and failover services fortraffic to the customer. More particularly, the process and systemallows for telecommunications network to utilize a common next-hop valueor identifier of announced border gateway protocol (BGP) routes toadvertise multiple routes to reach a destination customer network oraddress. By utilizing a common next-hop value or identifier in theannounced BGP information, the devices of the network may load balancecommunication packets to the destination customer or address among themultiple egress locations from the network, as well as providing fastfailover to alternate routes when a failure at the network or customeroccurs. In some instances, the next-hop value or identifier of theannounced BGP information may be shared among ingress points of agateway of the telecommunications network, common to a particularcustomer or customer network of the network, or common to a particularcustomer and ingress locations for a particular customer into thenetwork. Other uses and advantages of a routing based on a next-hop BGProuting information are discussed in more detail below.

FIG. 1 illustrates an exemplary operating environment 100 for providingmultiple egress points from a telecommunications network for a client ofthe network. In general, the environment 100 provides for establishingcommunication sessions between network users and for providing one ormore network services to network users. For example, content from acontent delivery network (CDN) may be provided to and/or from one ormore customers of the network through the operating environment 100discussed herein. With specific reference to FIG. 1, the environment 100includes an IP network 102, which may be provided by a wholesale networkservice provider. However, while the environment 100 of FIG. 1 shows aconfiguration using the IP network 102; it should be appreciated thatportions of the network may include non IP-based routing. For example,network 102 may include devices utilizing time division multiplexing(TDM) or plain old telephone service (POTS) switching. In general, thenetwork 102 of FIG. 1 may include any communication network devicesknown or hereafter developed.

The IP network 102 includes numerous components such as, but not limitedto gateways, routers, and registrars, which enable communication and/orprovides services across the IP network 102, but are not shown ordescribed in detail here because those skilled in the art will readilyunderstand these components. More relevant to this description is theinteraction and communication between the IP network 102 and otherentities, such as the one or more customer home or business local areanetworks (LANs) 106, where a user of the network will connect with thenetwork.

Customer network 106 can include communication devices such as, but notlimited to, a personal computer or a telephone 110 connected to arouter/firewall 114. Although shown in FIG. 1 as computer 110, thecommunication devices may include any type of communication device thatreceives a multimedia signal, such as an audio, video or web-basedsignal, and presents that signal for use by a user of the communicationdevice. The communication and networking components of the customernetwork 106 enable a user at the customer network 106 to communicate viathe IP network 102 to other communication devices, such as anothercustomer network 126 and/or the Internet 142. Components of the customernetwork 106 are typically home- or business-based, but they can berelocated and may be designed for easy portability. For example, thecommunication device 110 may be wireless (e.g., cellular) telephone,smart phone, tablet or portable laptop computer. In some embodiments,multiple communication devices in diverse locations that are owned oroperated by a particular entity or customer may be connected through theIP network 102.

The customer network 106 typically connects to the IP network 102 via aborder network 122, such as one provided by an Internet Service Provider(ISP). The border network 122 is typically provided and maintained by abusiness or organization such as a local telephone company or cablecompany. The border network 122 may providenetwork/communication-related services to their customers. In contrast,the communication device 120 accesses, and is accessed by, the IPnetwork 102 via a public switched telephone network (PSTN) 126 operatedby a local exchange carrier (LEC). Communication via any of the networkscan be wired, wireless, or any combination thereof. Additionally, theborder network 122 and PSTN 126 may communicate, in some embodiments,with the IP Network 102 through a media gateway device 130 or provideredge 132, 131. For ease of instruction, only three communication devices110, 115, 120 are shown communicating with the IP network 102; however,numerous such devices, and other devices, may be connected with thenetwork, which is equipped to handle enormous numbers of simultaneouscalls and/or other IP-based communications.

As mentioned above, the telecommunications network 102 may provide aninterface to a customer or border network through which the customer orborder network connects to the network. In one example, the interfacemay be a provider edge 132 or other type of edge device. Further, aborder network may connect to the telecommunications network throughmore than one provider edge devices in multiple locations. Inparticular, FIG. 2 is a schematic diagram illustrating a networkenvironment 200 to provide multiple routes for a client network 204through the network environment. The telecommunications network 202 ofFIG. 2 may be similar to that of FIG. 1 such that the networks mayinclude the same or similar components. Border networks 204, 224connected to the telecommunications network 202 may utilize the networkto send and receive packets of information to/from other customers tothe network, as described above.

As shown in FIG. 2, a border network 204 connects to the network 202through multiple provider edge devices. In particular, border network204 may include provider edge device A 206 that connects to the network202 through provider edge C 210 and provider edge device B 208 thatconnects to the network through provider edge D 212. In addition, bordernetwork 224 also connects to the network 202 through one or moreprovider edge devices. In particular, border network 224 includesprovider edge device G 220 that connects to the network 202 throughprovider edge E 216 and provider edge device H 222 that connects to thenetwork 202 through provider edge F 218. In one example, border network204 and border network 224 may be for the same customer to the networksuch that the border networks share the same autonomous system (AS)number identifying the border network to other telecommunicationsnetworks. As such, the particular border network 204, 224 may connect tothe telecommunications 202 network at multiple ingress/egress points invarious geographic locations and through various provider edge devicesor other edge devices. In other examples, border network 204 and bordernetwork 224 may be separately-managed border networks with different ASidentifiers. For the present disclosure, the border networks 204, 224are considered to include the same AS identifier such that, for routingpurposes, border network 204 and border network 224 may be considered aspart of the same border network or client.

The connections between the border network 204, 224 and the network 202provide multiple routes between the border network and thetelecommunications network 202 along which communications or packets maybe transmitted. It should be appreciated, however, that the bordernetwork 204, 224 may connect to the network 202 through any number ofedge devices in any geographic locations (known as “gateways”) in whichthe telecommunications network is present. For example, the customernetwork 204, 224 may connect to a first provider edge 210 located in SanFrancisco, Calif., and a second provide edge 216 located in Dallas, Tex.In still other examples, the border network 204 may include multipleconnections to the network 202 in a single gateway. Thus, the bordernetwork 204 may connect to the network 202 in San Francisco through bothprovider edge C 210 and provider edge D 212. Each provider edge deviceof the telecommunications network 202 may communicate with a similarprovider edge device of the border network 204, 224 to transmit andreceive communication packets between the networks. In this manner, aborder network 204, 224 may connect to the telecommunications network202 through multiple connections in multiple locations.

In one particular configuration, the network 202 may include a routereflector device or server 209. In general, the route reflector 209 is aserver or other type of networking device that receives an announcedroute from a telecommunications device and broadcasts the received routeto other devices in the network 202. As utilized in the network 202 ofFIG. 2, the route reflector 209 receives routes announced from provideredges 210-218 and broadcasts the received routes to other provideredges, servers, media gateways, etc. of the network 202. The use of theroute reflector 209 in the network 202 removes the need for each deviceof the network to communicate its announced route to each of the otherdevices in the network. Rather, the route reflector 209 announces eachreceived route to each other devices itself so that each device in thenetwork is not tasked with announcing routes to each other device,thereby reducing processing load on individual devices and overalltraffic on the network 202 to exchange routing information.

In one particular embodiment, the route that is received and broadcastby the route reflector 209 is a border gateway protocol (BGP) routeannounced by the transmitting provider edge 210-218. In general, BGPinformation (or BGP session, BGP feed or BGP data) is a table ofInternet Protocol (IP) prefixes which designate network connectivitybetween autonomous systems (AS) or separate networks. BGP informationfor a network route may include path (including next-hop information),network policies and/or rule-sets for transmission along the path, amongother information. The BGP feed may also include Interior GatewayProtocol (IGP) information for network routes within an AS or networkand/or other network information that pertains to the transmission ofcontent from the network. However, as described below, BGP informationmainly describes routes used by the network 202 to connect to externalnetworks or customers (such as border networks 204, 224) while IGPinformation describes routes through the network to connect one provideredge (such as provider edge E 216) to another provider edge (such asprovider edge C 210) through a telecommunications network.

One or more of the components of the network 202 may announce, through aBGP session, routes serviced by that component. For example, PE-C 210may provide a BGP announcement that indicates which networks (such asborder network 204) that may be accessed through PE-C. Thus, the BGPannouncement for PE-C 210 may include a path and next-hop informationthat designates a path along which packets may be transmitted orreceived from the connected networks (such as border network 204). Thenext-hop information generally identifies a particular device of thenetwork 202 through which a destination device or address is available.For example, a particular Internet Protocol (IP) address associated witha customer of the network 202 or border network 204 that is accessiblethrough the border network may be announced from PE-A 206 of the bordernetwork to PE-C 210 of the telecommunications network 202. PE-A 206 mayreceive the customer IP address from the customer itself as provided tothe border network 204 or the address may be provided to by the bordernetwork. Regardless of the origination of the customer IP address, PE-C210 receives the address and broadcasts the address to the network 202to indicate that communication packets intended for that IP address ordestination device may be processed through PE-A 206 to reach theintended destination.

In some instances, separate border networks may connect to thetelecommunications network 202 through the same edge device. Forexample, other border networks in addition to the border network 204illustrated may connect to the network 202 through PE-C 210. In such acircumstance, the BGP announcement from PE-C 210 may include routinginformation for both border networks. Regardless of the number of bordernetworks available behind the edge devices 210-218 of the network 202,the BGP announcements from the provider edges to the route reflector 214are re-broadcast to other components of the network 202 such that eachcomponent of the network 202 may become aware of the egress portsavailable for customers or networks connected to the network 202 and howto route packets intended for those connected networks.

As mentioned above, Border Gateway Protocol (BGP) is the routingprotocol used to distribute routes/prefixes between various networks.External BGP (eBGP) announcements are used to exchange routing betweennetworks with separate AS indicators and internal BGP (iBGP)announcements are used to announce routes internally within a network.Typically, an eBGP session is established between two edge or borderdevices in different networks (such as PE-A 206 of border network 204and PE-C 210 of network 202) and routes/prefixes are exchanged over theeBGP session. For eBGP learned prefixes, the receiving edge device setsa next hop address accompanying the announced route to either thephysical IP address or Loopback address (eBGP Multi-hop) of the eBGPneighbor. In many cases, there may be multiple connection points betweentwo networks (such as border network 204, 224 connecting through PE-C210 and through PE-E 216) which optimizes traffic, scales the networks,and provides geographic redundancy.

When routes are advertised internally (iBGP) within a network, the nexthop address of the advertising eBGP border network device (PE-A 206 inthis example) is advertised to all routers in the network 202 to allowthe routers to resolve the next hop address. One simple approach is toadvertise all routes inside a network with Next-Hop Self (NHS) set tothe originating border device 206. When prefixes are advertised with theNHS value, the network 202 routers rely on recursion on the next-hop toroute traffic to the proper edge device 206 for delivery to thedestination device. That is, next-hop values associated with BGPprefixes are resolved via the Internal Gateway Protocol (IGP).

Once BGP information is received at a router of the network 202, therouter may execute a selection process to determine which route toutilize to route a received communication packet to a destination deviceor address. In one embodiment, the route selection process executed bythe router or other network device selects the best BGP path forreaching a destination. Once the route is selected, the router willrecurse on the next-hop address in the IGP to determine how to routetraffic to the next-hop-address. In the event that a network routerreceives two prefixes with identical attributes or network costs intransmitting a packet along a particular route, the best path will beselected with the lowest associated cost, as discussed in more detailbelow.

For example, many networks utilize route reflector devices 209 forre-broadcasting received routes to external networks or customers. Suchroute reflectors 209 are often configured to only broadcast a preferredor “best” route to other devices in the network 202 for customers to thenetwork that have several egress routing paths. This acts to force alltraffic from the network to the customer on one preferred route,potentially overloading that route when other open routes are alsoavailable. However, the best route is often selected as being the lowestcost to the network 202 in terms of network usage or providing thehighest performance to the receiving device. Thus, the networkcongestion savings or customer experience for utilizing the selectedroute often may outweigh the potential for overloading of the selectedand advertised route. In another example, a failure at a border network,customer network, or network device may cause connecting network devicesto broadcast alternate routing information. In particular, if PE-G 220were to fail for any reason, border network 224 may rebroadcast a routethrough PE-H 222 as a backup route to reach the border network. Thisalternate routing may cause the route reflector 209 to re-calculate thebest path through the network 202 and re-broadcast the new routinginformation to the network devices. However, changes to the routinginformation may take a significant amount of time and processing power,during which communications intended for a destination network oraddress may be lost or otherwise dropped by the network 202.

As described, networks 202 that include a route reflector 209 device maycalculate a best path or route and provide that route to the routingdevices of the network 202. In particular, when a border network 204,224 to the network 202 has a single access route or port to/from thenetwork, the route reflector 209 announces the received BGP routinginformation for the single access route. However, in some circumstances,a border network has multiple connections to the network 202. In theexample illustrated in FIG. 2, route reflector 209 receives BGP routinginformation for border network 204, 224 from PE-C 210, PE-D 212, PE-E216, and PE-F 218 as that particular border network is connected to thenetwork 202 through each of the illustrated edge devices. In otherwords, packets or communications intended for border network 204, 224may be transmitted through any of noted edge devices. However, someroute reflectors 209 may be configured to only repeat or broadcast asingle route when several possible routes for a destination network ordevice are received at the route reflector. For example, the routereflector 209 of FIG. 2 may select PE-C 210 as the best route to reachborder network 204, 224 based on any number of cost metrics calculatedor determined by the route reflector. In particular, the route reflector209 may be configured to select a “best path” based on network metricsand broadcast the single “best path” to other network devices or routersin the network 202. Thus, each device in the network 202 may onlyreceive a single routing path through one of the possible multipleegress routes to the particular border network 204, 224. As such,traffic for a customer network connected to the telecommunicationsnetwork 202 may be limited to egress the network through a selected ordetermined “best path” or selected provider edge, reducing the potentialnetwork performance gained through multiple connections to the network.

Regardless of whether the best path, and in particular an egress deviceof the network 202, to reach a destination address or device is providedby a route reflector 209 or calculated by a network device, the next-hopvalue included in the BGP information broadcast through the networkincludes a Next-Hop Self (NHS) value that identifies a particular edgedevice for reaching the destination address. Thus, only one route istypically broadcast to reach the destination address such that alltraffic intended for the destination is processed through a singleegress location. Further, in some instances a failure may occur alongthe route (such as at the border network 204, 224 or at a customer site)that prevents packets for the destination from being terminated at thedestination address. Such failures may be detected at the networkthrough a BGP session failure, such as a session failure between PE-A206 of border network 204 and PE-C 210 of the telecommunications network202. In such circumstances, the selected edge device (PE-C 210)withdraws the prefixes in the BGP information transmitted to the routereflector 209 (or to other devices of the network 202) which causes theroute reflector to perform the BGP “best path” route selection algorithmagain and update/advertise these prefixes to the routers of the network.Depending on the size of the network, quantity of the prefixes, and thenumber of clients of the route reflector 209, this process may be a timeconsuming event during which communication packets intended for thedestination are transmitted to a “black hole” along the route andultimately lost. In networks without a route reflector device 209, a BGPNext-Hop change causes each router to perform the BGP route selectionprocess and to re-calculate a new best path for each prefix, which mayalso take a significant amount of time to converge the network into aroute to reach a destination network or device identified in the BGPinformation.

Through the best path routing described above, telecommunicationsnetwork 202 may not leverage the ability for border networks 204, 224 toconnect to the network in multiple locations or ingress points. Further,changes made to the routing tables due to a failure at a destinationdevice or other point along a route may cause significant issues inrouting packets to the intended destination. Thus, presented herein aremethods and systems to route communication packets in a network 202utilizing community Next-Hop values in announced BGP routes. ThisNext-Hop Anycast-type routing may provide a simple and flexible routingprocedure to load balance among the multiple connections to a bordernetwork 204, 224, as well as providing fast converging changes to therouting tables based on a failure along a selected or preferred route.In particular, FIG. 3 is a flowchart illustrating a method 300 for anedge device of a telecommunications network 202 to broadcast bordergateway protocol (BGP) and interior gateway protocol (IGP) announcementswith a community specific Next-Hop value that allows routers within thenetwork to load balance and respond quickly to failures in the network.Through the operations of the method 300 of FIG. 3, the network 202 mayprovide multiple routes through the network for a particular destinationdevice or address without significantly increasing the number of routesmaintained by the components of the network. In addition, the method 300allows for devices within the network 202 to load balance among manypossible routes of the network to reach the destination address ordevice, thereby reducing the load on any one particular route to thedestination.

Beginning in operation 302, the edge device to the network 202 (PE-C 210in this particular example) receives BGP information from an edge deviceof a border network (PE-A 206 of border network 204 in this example).The BGP information from PE-A 206 may be received during a BGP sessionbetween PE-A and PE-C 210 and includes information for connecting to theborder network 204. For example, the BGP information may include anautonomous system (AS) identifier and/or other route path informationthat the network 202 may utilize to determine which route to transmitpackets or communications intended for a destination address accessiblethrough the border network 204. Upon receiving the BGP information, PE-C210 may add next-hop information to the received BGP information andbroadcast the amended BGP information to other components of the network202, such as route reflector 209. In general, PE-C 210 provides BGPinformation to the route reflector 209 through a BGP session between thecomponents. Through this session, PE-C 210 provides BGP information thatincludes the received route and a next-hop identifier that is associatedwith PE-C, along with other routing parameters. Thus, in the aboveexample where the route reflector 209 is included in the network 202,each edge device 210-218 of the network connected to the border network204, 224 would similarly provide routing information to reach the bordernetwork through the associated edge devices. The route reflector 209, inturn, would select the “best route” to border network 204, 224 andbroadcast to the routers and other networking devices of the network 202the BGP information of the selected best route, including the next-hopvalue associated with the selected route. In the embodiment above wherethe route reflector 209 relays more than one route path or the network202 doesn't include a route reflector such that each component of thenetwork receives BGP information from each edge device 210-218 of thenetwork, each of the receiving components of the network execute analgorithm similar to that of the route reflector 209 to select the bestpath to reach the border network 204, 224. Regardless of the networkconfiguration, packets intended for a destination address associatedwith the border network 204, 224 may be funneled through one edge deviceof the network 202 if that device is part of the selected best pathwhile other connections to the destination address may go unused by thecomponents of the network 202 to reach the particular border network.

To more efficiently utilize each of the connections to the bordernetwork 204, 224 from the network 202, the method 300 of FIG. 3 includesadding a group-specific next-hop value in the announced BGP route inoperation 304. For example, each provider edge 210, 212 in a particulargateway location may be programmed or otherwise configured to provide agroup-specific next-hop value associated with a gateway into the network202 and for a particular customer or client of the network 202. One suchnext-hop value may be an IP address such as 1.1.1.1. However, it shouldbe appreciated that the next-hop value for a particular client orcustomer network may be any value transferable over a BGP session. Forexample, the next-hop value may be a community value associated with aparticular customer or destination device or a community value for aparticular customer in a particular gateway or location of the network202. Such next-hop values may be different than the gateway specificnext-hop value, such as IP address 1.1.1.2. In this manner, a customerconnecting to the network 202 in San Francisco may have a first next-hopvalue associated with San Francisco gateway to the network and may havea different next-hop value associated for other connections to thenetwork 202 in other areas or locations of the network, such as a secondnext-hop value associated with a Dallas gateway to the network.

With the group-specific next-hop value attached, each provider edge210-218 of the network 202 transmits the BGP route information receivedfrom the border network 204, 224 to the route reflector 209 in operation304, or alternatively broadcasts the BGP information to the othercomponents of the network 202. Thus, PE-C 210, PE-D 212, PE-E 216, andPE-F 218 would provide BGP route information that includes the receivedroute and the group-specific next-hop value to the route reflector 209.Further, each provider edge 210-218 transmitting the BGP route isconfigured to provide packets or communications from the network 202that utilize the group-specific next-hop value to the border network204, 224 associated with community next-hop value.

In some instances, the route reflector 209 relays the received BGP routeinformation to other components in the network 202. However, because therouting information received from each of the PEs 210-218 connected toborder network 204, 224 includes the same group-specific next-hop value,the components of the network 202 now have multiple paths in theirrouting table to reach the destination device or address. In otherwords, the routers of the network 202 may utilize any received routethat includes the group-specific next-hop value to reach a destinationaddress, allowing the devices to load balance among the multiple routes(assuming a similar or identical calculated cost to utilize theindividual routes) and to quickly failover packets from a droppedconnection to an active connection to the destination. Thus, the routereflector 209 does not select the best path to reach the border network204, 224 and only broadcasts that route. Rather, the route reflector 209broadcasts each received route with the group-specific next-hop value tothe components of the network 202 and each component may then determinewhich route to the border network 204, 224 to use for any particularreceived communication.

The group-specific next-hop value (also known as next-hop Anycast)provides a simple, flexible, and extensible load sharing and fastconverging enhancement for BGP routing within the network 202. Ingeneral, the community next-hop value may be based on any type ofcommunity of addresses or devices of the network 202. In one particularexample, a next-hop gateway (NHG) value may be used to share a next-hopvalue among the edge devices of a particular gateway to the network 202.Thus, every egress route to the border network 204, 224 from aparticular gateway (typically based on a geographical location of thegateway) may have the NHG applied to BGP announcements. In anotherexample, a next-hop peer (NHP) value may be used to share a next-hopvalue among the edge devices that connect to a particular customer ofthe network 202. In yet another example, a next-hop peer locality (NHL)value may be used to share a next-hop value among the edge devices thatconnect to a particular customer of the network 202 in one or moreparticular locations of the network 202. Although these three particularuses are described herein, it should be appreciated that the communitythat shares the next-hop value may be any type of community of thenetwork 202 and may include any number of edge or other networkingdevices or any number of egress routes to a destination network orcustomer.

For the next-hop gateway (NHG) embodiment discussed above, two or moreedge devices in the same gateway may broadcast BGP information thatincludes the same NHG value. More particularly, two or more edge devicesthat are connected to (or otherwise receive BGP announcements from) thesame border network 204 and are located in the same gateway (or samegeographical location) may utilize the NHG value in a similar manner asdescribed above. For example and turning to the network 200 of FIG. 2,PE-C 210 and PE-D 212 may both be located in a gateway located in SanFrancisco and connect to border network 204 through PE-A 206 and PE-B208, respectively. Although the gateway is defined here as being withinthe same location or metro area, it should be appreciated that a gatewayto the network 202 may be defined to include any number of deviceslocated in any geographic location of the network footprint. Generally,however, a gateway includes edge devices that are physically near eachother.

Rather than utilizing a next-hop self value that identifies theparticular edge device broadcasting the BGP information, PE-C 210 andPE-D 212 may insert a NHG value that is the same for the two edgedevices into the iBGP session announcements. In one particularembodiment, each prefix that advertised with an NHG is tagged with a BGP“community” that identifies it as a NHG prefix. The NHG is configured onthe edge device and advertised internally over iBGP sessions to thecomponents of the network 202, such as the route reflector 209 or othernetworking devices. Thus, the networking components of the network 202may receive multiple routes for a particular destination address thatinclude the same NHG value. This unique NHG value for the customerthrough a particular gateway allows the networking devices of thenetwork 202 to determine a shortest path to the destination address andtransmit packets along the selected route. However, because each edgedevice 210, 212 is located in the same gateway, the shortest path (orpath with the least “cost” to the network) is likely to be the same orsimilar for either route. The networking device may, in thiscircumstance, alternate sending packets between the two (or more) pathsto the edge devices of the gateway, thereby load balancing the trafficto the destination address between the edge devices 210, 212 of thenearest gateway to the networking device. In this manner, the shared NHGvalue provides multiple routing paths to reach a destination address fora device of the network 202 over which traffic may be load balancedacross, increasing the performance and capacity of thetelecommunications network.

In addition, the shared NHG value allows for a failover procedure withinthe network 202 that reduces routing traffic to a failed or unresponsiveborder network device 206, 208. In particular and returning to themethod 300 of FIG. 3, the edge device 210 may receive an indication of afailure along the route to the destination address in operation 306.This indication may be received during an eBGP session with PE-A 206 inwhich PE-A ceases to broadcast the availability of the destinationaddress through PE-A (as that link is now no longer functioning). Ingeneral, however, the indication may be received for any topology changein the route to the destination address. Upon receiving the indicationof the failure, the network edge device 210 may remove the NHG valuefrom the IGP information being broadcast in operation 308 and broadcastnew BGP information to the devices within the network 202 that does notinclude the NHG in operation 310. In other words, the edge device maycease utilizing the NHG value as it pertains to the affected destinationaddress. In response, the components of the network 202 may re-executethe algorithm to determine the shortest or best route to the destinationaddress. However, because the failed route no longer utilizes the NHG,the network components would select a different route to the bordernetwork 204 and may begin transmitting traffic for the destinationaddress to one of the remaining routes that does utilize or include theNHG. Large networks that utilize one route (such as a “best route” asdetermined by the route reflector 209) to a destination address may losetraffic to the border network 204 while the new routing information isbroadcast throughout the network 202. Thus, by utilizing the shared NHGvalue, the failover to the alternate route may occur faster to preventsignificant traffic loss to the failed route.

For example, assume PE-C 210 and PE-D 212 of the system 200 of FIG. 2are in or otherwise are associated with the same gateway of the network202. Thus, PE-C 210 and PE-D 212 may broadcast BGP information thatincludes the same NHG value for routes to border network 204, asdescribed above. If PE-A 206 fails, PE-C 210 would cease receiving BGPsession information from corresponding PE-A 206 such that PE-C maydetect the failure at PE-A. This change in network topology may betransmitted through the network 202, but may take some time, dependingon the size of the network. However, because PE-D 212 also announces aroute to the destination address (utilizing the NHG value), packetsintended for the destination address may be transmitted to PE-D 212 fortermination quickly. The use of the NHG value may thus reduce networkconvergence and brown out times associated with router or other devicefailures.

In another example, a next-hop peer (NHP) value may be used for eachconnection between the network 202 and a particular border network 204,224. For example, border network 204, 224 may connect to network 202 inseveral locations, such as through PE-C 210 and PE-D 212 in SanFrancisco and through PE-E 216 and PE-F 218 in Dallas. However, bordernetwork 204, 224 may be the same network identified by the same ASidentifier (i.e., controlled by the same administrator or entity andidentifiable to the telecommunications network 102 as the same networkbased on providing the same AS value in BGP sessions between the bordernetwork and the telecommunications network). In this example, each edgedevice 210-218 of the network 202 that connects to a particular bordernetwork 204, 224 may utilize a unique NHP value when broadcastingrouting information to the other components of the network. The use ofthe advertised routes by the components of the network 202 occurs asdescribed above. In particular, each component (or a route reflector 209component) determines which received route is the shortest or best pathand utilizes that route to transmit traffic to the destination address,while also load balancing between routes that have identical or similarcosts. For example, routers of the network 202 that are near SanFrancisco may load balance between PE-C 210 and PE-D 212 to reach bordernetwork 204, but would not transmit traffic to PE-E 216 in Dallas aslong as the routes through San Francisco remain active. In oneparticular implementation, the border network 204, 224 would advertisethe NHP value rather than have it assigned by the edge device 210-218 ofthe network 202. The NHP value may be provided by the network 202 or anetwork administrator to the border network 204, 224 for announcingduring eBGP sessions between the networks.

A failover procedure may also be provided through the use of the NHPvalue. For example, edge devices 210-218 of the network 202 may utilizethe same NHP value as discussed above. If PE-A 206 fails, PE-C 210 wouldcease receiving BGP session information such that PE-C may detect thefailure at PE-A. This change in network topology may be transmittedthrough the network 202 to reroute traffic for the intended address.However, because PE-E 216 in Dallas also announces a route to thedestination address (utilizing the NHP value), packets intended for thedestination address may be transmitted to PE-E 216 for termination. Thismay aid in providing a robust network for a customer as the customer mayonly experience a failure or outage in a particular location, such asSan Francisco. Rather than losing all traffic from the network 202,communication packets may be re-directed through another connection tothe network. This may be especially helpful when the supporting network(border network 204, 224) remains active, but one or more of thecustomer devices go into a failure state. By quickly re-directingtraffic to an active portion of the customer network, less traffic maybe lost due to the failure at the customer's devices.

In yet another example, a local next-hop peer (NHL) value may be usedfor each connection between the network 202 and a particular bordernetwork 204, 224 in particular locations. For example, many customers orborder networks 204, 224 have traffic engineering requirements orcriteria that route traffic with specific prefixes to a specific set ofedge devices of the border network, such as when a particularapplication (for example, web services) is available only fromapplication servers in a particular geographic location. Thus, thecustomer may prefer to have all traffic intended for the particularapplication provided to the border network 204, 224 in the sameparticular geographic location. In general, any traffic engineeringconsiderations of the customer may lead to the use of the NHL value. Inthis circumstance, the NHL value may allow the routing of traffic to aborder network 204, 224 through a subset of connection points betweenthe border network and the telecommunications network 202.

Utilizing the network 200 of FIG. 2 as an example of the use of an NHLvalue, assume that border network 204, 224 connects to network 202 inseveral locations, such as through PE-C 210 and PE-D 212 in SanFrancisco and through PE-E 216 and PE-F 218 in Dallas. However, theborder network (or a connected customer to the border network) mayintend for all traffic to a specific destination address be transmittedthrough the Dallas located edge devices PE-E 216 and PE-F 218 and notthrough PE-C 210 or PE-D 212. In this example, the edge devices 216-218of the network 202 that connects to the border network 204, 224 inDallas may utilize an NHL value when broadcasting routing informationinto the network. In addition, the border network 204, 224 may utilize adifferent next-hop value, such as the customer-specific NHP discussedabove, to advertise customer destinations for other connections betweenthe border network and the telecommunications network 202 (in thisexample, through PE-C 210 or PE-D 212). Thus, two different next-hopvalues may be used by the border network 204, 224 for connection to acustomer, such as a global NHP value and a local NHL value for trafficintended for a specific destination address.

Similar to above, each component (or a route reflector 209 component) ofthe network 202 determines which received route is the shortest or bestpath and utilizes that route to transmit traffic to the destinationaddress, while also load balancing between routes that have identical orsimilar costs. Thus, routers of the network 202 near San Francisco maycontinue to provide general traffic to the customer through the nearbyedge devices 210-212 in San Francisco based on the advertised NHP value.However, for traffic intended for a specific application or destinationaddress of the customers, the routers may rely on the NHL value and sendtraffic through the Dallas-based edge devices 216-218. Theapplication-specific traffic may be load balanced between edge devices216-218 as described above, as well as providing a failover option forfailures along a route to the customer. Although described as being inthe same location, it is not necessary that the edge devices utilizingthe NHL value be near each other. Rather, any connection point betweenthe border network 204, 224 and the telecommunications network 202 mayutilize the NHL value. The NHL value thus operates to create a subset ofconnection points to the destination customer through which specifictraffic may be steered in response to traffic engineering considerationsof the border network or customer to the network.

FIG. 4 is a flowchart illustrating a method 400 for a networking device,such as a router, of a telecommunications network 202 to transmit apacket to a customer network 204 based on a customer-specific value as anext-hop value. The operations of the method 400 may be performed by arouter or any other component of the network in response to receiving apacket or communication intended for a destination address. Such acommunication may be provided by a component within the network 202(such as a content server) or from a telecommunications device externalto the network (such as a content server of a provider network).

Beginning in operation 402, the router receives a BGP announcement thatincludes multiple egress routes to a customer network that includes thegroup-specific next-hop value as described above. In one example, theBGP route information is received at router from route reflector 209 asannounced from an edge device connected to a border network 204, 224.The router may store or otherwise maintain the received BGP route withthe community-specific next-hop value for routing of communications tothe customer network.

In operation 404, the router may select a preferred route through thenetwork 202 from the multiple egress routes known to the border network204, 224. For example, through the received information, router may beaware of four egress points from the network 202 to the border network204, 224, namely PE-C 210, PE-D 212, PE-E 216, and PE-F 218, based oneach of the edge devices advertising routes with the group-specificnext-hop value. Thus, the router selects one of the multiple routes tothe border network 204, 224 to transmit a received packet. In oneembodiment, the router may select a “best route” based on any type ofnetwork performance metric and transmit the packet along the selectedroute. In another embodiment, the router may load balance among theknown egress points to the border network 204, 224 and may select theegress route accordingly

The router receives a packet or communication intended for the bordernetwork 204, 224 in operation 406. As mentioned, the packet may includerouting information that identifies the destination address and/or anext-hop value associated with the border network 204, 224. In operation408, the router selects the determined best route through the network202 from the multiple egress routes known to the border network 204, 224and, in operation 410, routes the received packet to the border networkalong the selected route.

FIG. 5 is a block diagram illustrating an example of a computing deviceor computer system 500 which may be used in implementing the embodimentsof the components of the network disclosed above. For example, thecomputing system 500 of FIG. 5 may be the edge device discussed above.The computer system (system) includes one or more processors 502-506.Processors 502-506 may include one or more internal levels of cache (notshown) and a bus controller or bus interface unit to direct interactionwith the processor bus 512. Processor bus 512, also known as the hostbus or the front side bus, may be used to couple the processors 502-506with the system interface 514. System interface 514 may be connected tothe processor bus 512 to interface other components of the system 500with the processor bus 512. For example, system interface 514 mayinclude a memory controller 514 for interfacing a main memory 516 withthe processor bus 512. The main memory 516 typically includes one ormore memory cards and a control circuit (not shown). System interface514 may also include an input/output (I/O) interface 520 to interfaceone or more I/O bridges or I/O devices with the processor bus 512. Oneor more I/O controllers and/or I/O devices may be connected with the I/Obus 526, such as I/O controller 528 and I/O device 540, as illustrated.

I/O device 540 may also include an input device (not shown), such as analphanumeric input device, including alphanumeric and other keys forcommunicating information and/or command selections to the processors502-506. Another type of user input device includes cursor control, suchas a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to the processors 502-506and for controlling cursor movement on the display device.

System 500 may include a dynamic storage device, referred to as mainmemory 516, or a random access memory (RAM) or other computer-readabledevices coupled to the processor bus 512 for storing information andinstructions to be executed by the processors 502-506. Main memory 516also may be used for storing temporary variables or other intermediateinformation during execution of instructions by the processors 502-506.System 500 may include a read only memory (ROM) and/or other staticstorage device coupled to the processor bus 512 for storing staticinformation and instructions for the processors 502-506. The system setforth in FIG. 5 is but one possible example of a computer system thatmay employ or be configured in accordance with aspects of the presentdisclosure.

According to one embodiment, the above techniques may be performed bycomputer system 500 in response to processor 504 executing one or moresequences of one or more instructions contained in main memory 516.These instructions may be read into main memory 516 from anothermachine-readable medium, such as a storage device. Execution of thesequences of instructions contained in main memory 516 may causeprocessors 502-506 to perform the process steps described herein. Inalternative embodiments, circuitry may be used in place of or incombination with the software instructions. Thus, embodiments of thepresent disclosure may include both hardware and software components.

A machine readable medium includes any mechanism for storing ortransmitting information in a form (e.g., software, processingapplication) readable by a machine (e.g., a computer). Such media maytake the form of, but is not limited to, non-volatile media and volatilemedia. Non-volatile media includes optical or magnetic disks. Volatilemedia includes dynamic memory, such as main memory 516. Common forms ofmachine-readable medium may include, but is not limited to, magneticstorage medium (e.g., floppy diskette); optical storage medium (e.g.,CD-ROM); magneto-optical storage medium; read only memory (ROM); randomaccess memory (RAM); erasable programmable memory (e.g., EPROM andEEPROM); flash memory; or other types of medium suitable for storingelectronic instructions.

Embodiments of the present disclosure include various steps, which aredescribed in this specification. The steps may be performed by hardwarecomponents or may be embodied in machine-executable instructions, whichmay be used to cause a general-purpose or special-purpose processorprogrammed with the instructions to perform the steps. Alternatively,the steps may be performed by a combination of hardware, software and/orfirmware.

Various modifications and additions can be made to the exemplaryembodiments discussed without departing from the scope of the presentinvention. For example, while the embodiments described above refer toparticular features, the scope of this invention also includesembodiments having different combinations of features and embodimentsthat do not include all of the described features. Accordingly, thescope of the present invention is intended to embrace all suchalternatives, modifications, and variations together with allequivalents thereof.

We claim:
 1. A telecommunications network comprising: a first networkedge device; a second network edge device; and a border network incommunication with a telecommunications network, wherein the firstnetwork edge device establishes a border gateway protocol (BGP) sessionwith a first edge device of the border network, the BGP sessionproviding first routing information for transmitting communicationpackets to a destination address through the first network edge deviceof the border network, the second network edge device receives secondrouting information from a second edge device of the border network, thesecond routing information for transmitting communication packets to thedestination address through the second network edge device of the bordernetwork, and the first network edge device and second network edgedevice transmit internal BGP information to at least one device of thetelecommunications network, the internal BGP information comprising thedestination address and a group-specific next-hop identifier.
 2. Thetelecommunications network of claim 1, wherein the first network edgedevice and the second network edge device are located in the samegateway of the telecommunications network.
 3. The telecommunicationsnetwork of claim 1, wherein the group-specific next-hop identifier is anext-hop peer identifier and the first network edge device and thesecond network edge device are connected to the border network indifferent geographical locations, the border network associated with aparticular autonomous system (AS) number border network of thetelecommunications network.
 4. The telecommunications network of claim3, further comprising: a third network edge device receiving thirdrouting information from a third edge device of the border network incommunication with the telecommunications network, wherein the thirdnetwork edge device transmits internal BGP information to the at leastone device of the telecommunications network comprising the destinationaddress and a next-hop peer local identifier, the next-hop peer localidentifier different than the next-hop peer identifier and based atleast on a customer network configuration.
 5. The telecommunicationsnetwork of claim 1, wherein the at least one device of thetelecommunications network receiving the internal BGP information is aroute reflector networking device.
 6. The telecommunications network ofclaim 1, wherein an indication of a failure of the first edge device ofthe border network in communication with the telecommunications networkis received, and the communication packets are transmitted to thedestination address through the second network edge device of the bordernetwork in response to the indication of the failure of the first edgedevice of the border network.
 7. The telecommunications network of claim1, wherein a first portion of the communication packets are transmittedto the destination address through the first network edge device of thetelecommunications network, and a second portion of the communicationpackets are transmitted to the destination address through the secondnetwork edge device of the telecommunications network to load balancethe communication packets between the first network edge device and thesecond network edge device.
 8. The telecommunications network of claim2, wherein the gateway comprises networking devices from two or morecustomers of the telecommunications network.
 9. The telecommunicationsnetwork of claim 1, wherein the at least one device of thetelecommunications network utilizes the group-specific next-hopidentifier received from the first network edge device and the secondnetwork edge device to calculate an optimal route through thetelecommunications network to destination address.
 10. Thetelecommunications network of claim 1, wherein the group-specificnext-hop identifier is utilized by the telecommunications network totransmit communication packets to the border network.
 11. A networkingdevice comprising: at least one communication port for receiving routinginformation associated with a telecommunications network; a processingdevice; and a computer-readable medium connected to the processingdevice configured to store information and instructions that, whenexecuted by the processing device, performs the operations of:establishing, utilizing a first network edge device, a border gatewayprotocol (BGP) session with a first edge device of a border network incommunication with a telecommunications network, the BGP sessionproviding first routing information for transmitting communicationpackets to a destination address through the first network edge deviceof the border network; receiving, utilizing a second network edgedevice, second routing information from a second edge device of theborder network in communication with the telecommunications network, thesecond routing information for transmitting communication packets to thedestination address through the second network edge device of the bordernetwork; and transmitting internal BGP information from the firstnetwork edge device and the second network edge device to at least onedevice of the telecommunications network, the internal BGP informationcomprising the destination address and a group-specific next-hopidentifier.
 12. The networking device of claim 11, wherein the firstnetwork edge device and the second network edge device are located inthe same gateway of the telecommunications network.
 13. The networkingdevice of claim 11, wherein the group-specific next-hop identifier is anext-hop peer identifier and the first network edge device and thesecond network edge device are connected to the border network indifferent geographical locations, the border network associated with aparticular autonomous system (AS) number border network of thetelecommunications network.
 14. The networking device of claim 13,wherein the information and instructions, when executed by theprocessing device, further perform the operations of: receiving,utilizing a third network edge device, third routing information from athird edge device of the border network in communication with thetelecommunications network; transmitting internal BGP information fromthe third network edge device to the at least one device of thetelecommunications network comprising the destination address and anext-hop peer local identifier, the next-hop peer local identifierdifferent than the next-hop peer identifier and based at least on acustomer network configuration.
 15. The networking device of claim 11,wherein the at least one device of the telecommunications networkreceiving the internal BGP information is a route reflector networkingdevice.
 16. The networking device of claim 11, wherein the informationand instructions, when executed by the processing device, furtherperform the operations of: receiving an indication of a failure of thefirst edge device of the border network in communication with thetelecommunications network; and transmitting the communication packetsto the destination address through the second network edge device of theborder network in response to the indication of the failure of the firstedge device of the border network.
 17. The networking device of claim11, wherein the information and instructions, when executed by theprocessing device, further perform the operations of: transmitting afirst portion of the communication packets to the destination addressthrough the first network edge device of the telecommunications network;and transmitting a second portion of the communication packets to thedestination address through the second network edge device of thetelecommunications network to load balance the communication packetsbetween the first network edge device and the second network edgedevice.
 18. The networking device of claim 12, wherein the gatewaycomprises networking devices from two or more customers of thetelecommunications network.
 19. The networking device of claim 11,wherein the at least one device of the telecommunications networkutilizes the group-specific next-hop identifier received from the firstnetwork edge device and the second network edge device to calculate anoptimal route through the telecommunications network to destinationaddress.
 20. The networking device of claim 11, wherein thegroup-specific next-hop identifier is utilized by the telecommunicationsnetwork to transmit communication packets to the border network.